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Abstract — Maximum Likelihood (ML) decoding is the optimal 
decoding algorithm for arbitrary linear block codes and can be 
written as an Integer Programming (IP) problem. Feldman et al. 
relaxed this IP problem and presented Linear Programming (LP) 
based decoding algorithm for linear block codes. In this paper, we 
propose a new IP formulation of the ML decoding problem and 
solve the IP with generic methods. The formulation uses indicator 
variables to detect violated parity checks. We derive Gomory cuts 
from our formulation and use them in a separation algorithm to 
find ML codewords. We further propose an efficient method of 
finding cuts induced by redundant parity checks (RPC). Under 
certain circumstances we can guarantee that these RPC cuts are 
valid and cut off the fractional optimal solutions of LP decoding. 
We demonstrate on two LDPC codes and one BCH code that 
our separation algorithm performs significantly better than LP 
decoding. 

Index Terms — ML decoding, LP decoding, Integer program- 
ming, Separation algorithm. 



I. Introduction 

LOW-DENSITY PARITY-CHECK (LDPC) codes have 
attracted significant interest in the research community in 
the last decade. LDPC codes are generally decoded by Belief 
Propagation (BP) (or Sum-Product) algorithm. BP exploits the 
sparse sttucture of the parity check matrix of LDPC codes 
very well and achieves good performance. However, due to 
the heuristic nature of BP algorithm, it is not possible to 
guarantee the performance of BP decoders at very low error 
rates. Moreover, the performance of BP is very poor for 
arbitrary linear block codes with dense parity check matrices 
(which means that the corresponding Tanner graph contains 
short cycles). 

ML decoding of linear block codes can be modeled as 
an IP problem. However, since the ML decoding is NP-hard 
iffl , solving this IP problem is computationally feasible only 
for small instances. Nevertheless considering ML decoding as 
an IP problem yields a new approach to derive sub-optimal 
algorithms. These algorithms offer some advantages compared 
to BP decoding. First, these approaches rely on a well-studied 
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mathematical theory which enables quantitative statements 
(e.g. convergence, complexity, correctness, etc.) with regard to 
the decoding process and its result Jfj), iflOl . |fl3ll . Secondly, 
they are not limited to sparse matrices. 

In IflOl Feldman et al. proposed a new algorithm based on 
LP to decode binary linear codes. This LP decoding algorithm 
utilizes a set of constraints which contains all valid codewords 
of a given code and a linear objective function. Minimizing 
this objective function over the resulting polytope yields the 
ML codeword if the optimal solution is integral (known as 
ML certificate property |10|). If the optimal solution is not 
integral then LP decoder outputs an error. 

Recently, LP decoding has been improved towards lower 
complexity (H, (5), 0JJ, 03), ED, lEESI ) and better perfo- 
rmance (J3), ||4), (SJ, (9)). Analysis of error correction perfor- 
mance of LP decoding (Q, ATI . Ifl6ll ) and the relationship 
to iterative message passing algorithms ( IflOl . fi"5l . ifTTl ) have 
also been studied in the literature. 

In this paper, we concentrate on improving linear program- 
ming decoding using a separation algorithm. We inttoduce an 
alternative IP formulation for the decoding problem. Instead of 
solving the optimization problem, we attempt to find the ML 
solution by an iterative separation approach: First, we relax the 
IP formulation and solve the resulting linear program. In case 
of a non-integral optimal solution, we derive inequalities which 
cut off this non-integral solution, add these inequalities to 
the LP formulation and resolve the LP problem. This process 
continues until an optimal integer solution is found or further 
cuts cannot be generated. It should be noted that this general 
integer programming approach known as separation problem 
has first been applied to LP decoding by Taghavi and Siegel 
fT3l . Our approach offers however the following advantages 
which remarkably facilitate LP based decoding. 

1) The number of constraints in the new IP formulation 
is the same as the number of rows in the parity check 
matrix. Each parity check equation which is originally 
in GF(2) is converted into a linear constraint in R™ by 
means of an auxiliary variable. 

2) The auxiliary variables serve as indicators which can be 
used for identifying violated parity check constraints. 
We can prove that we detect violated inequalities faster 
than the adaptive algorithm of Taghavi and Siegel under 
some mild assumptions. 

3) We formally show that the Forbidden Set Inequalities 
[8 1 are a subset of the set of Gomory cuts (see lfl2l ) 
which can be deduced from our formulation. 

4) We provide empirical evidence that our new separation 
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algorithm performs better than LP decoding. This is 
mainly due to generating strong cuts efficiently using 
alternative representations of the codes at hand. 
To provide empirical evidence we applied the New Separa- 
tion Algorithm to decode two LDPC codes along with one 
BCH code. 

The rest of this paper is organized as follows. We introduce 
notation in Section II and briefly review relevant literature 
in Section III. In Section IV, we introduce the new IP 
formulation, its LP relaxation, and the New Separation 
Algorithm . In Section V we present our numerical results 
and compare them with BP, LP decoding, and the lower bound 
resulting from ML decoding. The paper is concluded with 
some remarks and further research ideas in Section VI. 

II. Notation and background 

A binary linear block code with cardinality 2 k and block 
length n is a k dimensional subspace of the vector space 
{0, 1}™ defined over the field GF{2). The linear code C is 
given by k basis vectors of length n which are represented by 
a k x n matrix G (generator matrix). Equivalently C can be 
described by a parity check matrix H G {0, l} mx " where 
to = n — fc.We thus have x G C, i.e. i is a codeword, 
if and only if Hx = in GF(2). We denote the i th row 
and j th column of H by Hi,., H. j respectively. Hi^.x = 
in GF(2) is defined as the i th parity check constraint. The 
index set / = {1. . . . , to} refer to the rows and the index set 
J = {1, . . . ,n} refer to the columns of H. The matrix H is 
often represented by a Tanner graph G = (V, E). The node set 
V of G consists of the two disjoint node sets indexed by / and 
J called the check nodes and variable nodes respectively. An 
edge G E connects node i and j if and only if = 1. 

The ML decoding problem for any binary code C G {0, 1}™ 
can be written in terms of the mathematical program 

min{c T x : x G C} = min{c X : x G conv(C)}. (1) 

Here, c G R™ is the cost vector obtained by the log- 
likelihood ratios q = log ( ^rj—E^ J f° r a given received 
bit ii and conv(C) denotes the convex hull of C i.e. the 
codeword polytope. The left hand side of the equation (fTJ 
is an integer programming problem which is known to be 
NP-hard [1]. Replacing C with conv(C) leads to a linear 
programming problem which is stated on the right hand 
side of £[). Although linear programming is polynomially 
solvable in general, computing conv(C) is intractable. In other 
words a concise description of conv(C) by means of linear 
inequalities increases exponentially in the block length n. 
Thus ML decoding remains a challenging task. Nevertheless, 
linear programming decoding can be applied efficiently if 
good approximations of the codeword polytope can be found. 
Recently attempts in this direction have been made, (e.g. 1 5 1 , 

Eni, na, ma, mi 

Feldman et al. |flj}| introduced the LP decoder which 
minimizes c T x over a relaxation of the codeword polytope. 
The relaxation is achieved by using the parity check matrix 
H. Each row (check node) i 6 I defines a local code Ci, i.e. 
local codewords x G C, are the bit sequences which satisfy 



the i parity check constraint. Note that C — C\ f~l . . . n C rn . 

Lemma 2.1 ( Kl4\l ): Let P = conv(Ci) n . . . nconv(C m ). If 
C = Ci n . . . n C m then conv(C) C P. 

P is generally referred to as the fundamental polytope 
(El, H2, 02)). This relaxation has the advantage that the 
complexity of describing the convex hull of any local code 
conv(Ci) and thus of P is much less than the complexity of 
describing the codeword polytope C. The LP decoder solves 
the problem min{c T x : x e P}. 

Several approaches are used in ll5l. ifTOl. lull. Ifl4l lfl9l 
to write constraints completely describing P. We are going 
to use the set of constraints already introduced in [ 1 1 and 
referred to as Forbidden Set Inequalities in [8|. The index set 
of variable nodes which are adjacent to check node i is defined 
as Ni := {j G J : H$j = 1}. Using S C JV, we assign values 
to code bits Xj as follows. Set Xj — 1 for all j G S, and 
Xj = for all j E Ni\ S. For j Ni, Xj can be chosen 
arbitrarily. These value assignments to variables are feasible, 
i.e. satisfy the parity check constraint, for the local code Ci 
if \S\ is even. If |5| is odd, they are, however, infeasible or 
forbidden. From this observation the so called Forbidden Set 
Inequalities are derived. Let Sj = {S C Ni : \S\ odd}. It is 
shown in iflOl that conv(Ci) can be described by 

x i +520- - x i) ^ iVSGSi (2) 

j£Ni\S jes 

which can equivalently be written as 

^Xj- X J <\S\-1VS € Si. (3) 

jes jem\s 

Consequently the LP decoder solves 

minc T a; (LPD) 

s.t. xj — xj < 1 5 1 — 1 V5 G Si, i = 1, . . . , m 

jes jeNi\s 
0<x<l. 

If LPD has an integral optimal solution then the LP decoder 
outputs the ML codeword. If LPD has a non-integral optimal 
solution then the LP decoder outputs an error. The number of 
Forbidden Set Inequalities induced by check node i is 2 5 ^~ 1 
where S(i) — Y^=i^ij ^ s tne cnec k node degree, i.e. the 
number of edges incident to node i. The LP decoder can thus 
be applied successfully to low density codes. As the check 
node degrees increase the computational load of building and 
solving the LP model is however in general prohibitively 
large. This makes the explicit description of the fundamen- 
tal polytope via Forbidden Set Inequalities inapplicable for 
high density codes. To overcome this difficulty an alternative 
formulation which requires 0(n 3 ) constraints is proposed in 
[ 1 1 . More recent formulations of |2] and lfl9l have size 
linear in the length and check node degrees. Another approach 
applicable to high density codes is to solve the corresponding 
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separation problem of LPD [ 13 1. The separation problem over 
an implicitly given polyhedron is defined as follows: 

Definition 2.2: Given a bounded rational polyhedron P C 
R n and a rational vector x* E R", either conclude that x* E P 
or, if not, find a rational vector (II, n ) E R™ x R such that 
Tl T x < n and U T x < Tl T x* for all x E P. In the latter case 
(II, IIo) is called a valid cut. 

In separation algorithms (see lfl2l ) one iteratively computes 
families A of valid cuts until no further cuts can be found. In 
the separation algorithm of 1131 . which is called adaptive LP 
decoding by the authors, Forbidden Set Inequalities are not 
added all at once in the beginning as in iflOl but iteratively. 
In other words, the separation problem for the fundamental 
polytope is solved by searching violated Forbidden Set In- 
equalities. In the initialization step of the LP min {c T x : < 
x < 1} is computed. An optimal solution x* is checked in 
0(m6 max + nlogn) time, if x* violates any forbidden set 
inequality where S max is the maximum check node degree. 
If some of the Forbidden Set Inequalities are violated then 
these inequalities are added to the formulation and the LP is 
resolved including the new inequalities. 

Adaptive LP decoding stops when the current optimal 
solution x* satisfies all Forbidden Set Inequalities. If x* is 
integral then it is the ML codeword otherwise an error is 
output. Note that putting the LP decoder in an adaptive setting 
does not yield an improvement in terms of frame error rate 
since the same solutions are found. On the other hand the 
adaptive LP decoder converges with less constraints than the 
LP decoder which has a positive effect on computation time. 

The communication performance of LP decoding motivated 
researchers to find better approximations of the codeword 
polytope as part of ML decoding. One way is to tighten 
the fundamental polytope with new valid inequalities. Among 
some other generic techniques of cut generation, adding so 
called RPC cuts is proposed in ifTOl . Redundant parity checks 
are obtained by adding a subset of rows of H matrix in GF(2). 
These checks are redundant in the sense that they do not alter 
the code (they may even degrade the performance of BP 1 10|). 
However they induce new constraints in the LP formulation 
which may cut off a particular non-integral optimal solution 
thus tightening the fundamental polytope. An open problem is 
to find methods to generate redundant parity checks efficiently 
such that the induced constraints are guaranteed to cut off a 
non-integral LP solution. 

To the best of our knowledge two approaches for generating 
potential cuts exist so far. First, adding redundant parity check 
cuts which result from adding any two rows of H |10|. 
Secondly, the approach in lfl"3"l which makes use of the cycles 
in the Tanner graph: 1) given a non-integral optimal solution 
x* remove all variable nodes j form the Tanner graph for 
which x* is integral; 2) find a cycle by randomly walking 
through the pruned Tanner graph; 3) add the rows of the H 
matrix in GF(2) which correspond to the check nodes in the 
cycle; 4) check if the found RPC introduces a cut. 



III. A New Separation Algorithm Based on an 

ALTERNATIVE IP FORMULATION 

Our separation algorithm is based on the following formu- 
lation which we refer to as Integer Programming Decoding 
(IPD). 

minc T a; (IPD) 
s.t. Hx - 2z = 
x e {0, 1}" 
z > 0, integer 

IPD is an integer programming problem which works as an 
ML decoder. The auxiliary variable z E Z m ensures the binary 
constraint Hx = over GF(2) turns into a constraint over 
the real number field R which is much easier to handle. This 
formulation has the additional advantage that the number of 
constraints is the same as the number of rows of the parity 
check matrix. Note that LPD can also be used as an ML 
decoder by restricting x to be in {0, 1}". Yet in this case the 
number of constraints is exponential in the check node degree. 
Although our formulation IPD has less constraints, this does 
not change the fact that ML decoding is NP-hard. Therefore 
our approach is to solve the separation problem by iteratively 
adding new cuts H T x < n according to Definition 12.21 and 
solving the LP relaxation of IPD given by 

minc T a; (RIPD) 
s.t. Hx - 2z = 

n T a;<no (n,n )eA 

< x < 1 
z > 0. 

Note that in the initialization step there are no cuts of type 
Il T x < IIo i e. A = 0. If RIPD has an integral solution 
(x*,z*) e Z" +m then x* is the ML codeword. Otherwise we 
generate cuts of the type II T a; < IIo m order to exclude the 
non-integral solution found in the current iteration. We add 
these inequalities to the formulation and solve RIPD again. 
In a non-integral solution of RIPD x or z (or both) is non- 
integral. If x G Z" and z E R m \Z m then we add Gomory cuts 
(see [ 12 1) which is a generic cut generation technique used in 
integer programming. Surprisingly, in this case Gomory cuts 
can be shown to correspond to Forbidden Set Inequalities. 

Theorem 3.1: Let {x* , z*) E Z n x R m be the optimal 
solution of RIPD such that z* eI\Z for i E I. Then the 
Gomory cut which is violated by (x*, z*) is the Forbidden Set 
Inequality 

5>i- E ^<isi-i (4) 

jes jeN,\s 

where S := {j E N 4 | x* = l}. 
Proof: 

We apply the general method known as Gomory's cutting 
plane algorithm (see e.g. lTT2l ) to our special case. Gomory 
cuts are derived from the rows of the simplex tableau in 
order to cut off non-integral LP solutions and find the optimal 
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solution to the integer linear programming problems. Consider 
RIPD at any step: 

minc T a; (RIPD) 
s.t. Hx - 2z = 
< x < 1 
Ax < b 
z>0 

where c, x 6 K™, H E {0,l} mx ™, z E M m , A E 
{-1, 0, l} Axn for some A G No and b E N A . Note that A is the 
number of constraints added iteratively until the current step, 
i.e. A = | A|. The A x n matrix A is the coefficient matrix of the 
iteratively added constraints, i.e. H T x < Uq (II, IIo) E A. 
We denote the right hand sides of these constraints with the 
vector b. RIPD in standard form can be written as follows: 

minc T a; (RIPD) (5) 

s.t. z-Hx = (6) 

x + si = l (7) 

Ax + s 2 = b (8) 

z > 0, x > 0, s > 0. (9) 

where H := \H, s = (si, S2) E M" +A . For ease of notation 
we rewrite (H}-© as 

mhic T y (10) 

sX.Py = q (11) 

V > 0. (12) 

Note that 

C (c± j ■ ■ ■ j C m , C m +i , . . . , C m + n , C m + n +i , . . . , C m -j-2?i + A ) 

= (0, . . . ,0,ci, . . . ,c„,0, . . . ,0), 

1 Vm 1 Um,-\-l 1 ■ • ■ 1 ym-\-ri ? ?/m+n+l ym-\-2n-\-X ) 

= (zi,...,z m ,x 1 ,...,x n ,sx,...,s n+ x) and 
q = ( qi , q m ,q m+1 , . . . , q m+n , q m +n+l, ■ ■ ■ 
= (0,. ..,0,1,. ..,l,6i,. ..,6 A ). 

The constraint matrix P has m + n + A rows and m + 2n + A 
columns. We denote the a th row of P with P Q where a E 
{l,...,m + n + A} and (3 th column of P with P 13 where 
(3 E {1, . . . , m + 2n + A}. The component in row a and 
column j3 is denoted with P a p. Additionally, we define the 
a th unit vector as e a E R" l +"+ A . Thus, we rewrite P as 

p _ j^gl e mpm+l pm+n^m+n+l g m+2n+Aj 

The first m columns of the constraint matrix P are the unit 
vectors corresponding to the variables {zx . . . z m }. Likewise, 
the last n + A columns are the unit vectors corresponding to 
the slack variables {si . . . s n+ \}. 

The first m linear equations of Py = q are of the form: 

Zi ■ £j = for all i E {1, . . . , m}. 



Let ?/* = (z*,x*,s*) E W n+2n+x be the optimal solution to 
©-(O. By assumption it is x* E {0, 1}™. For i e {1, . . . , m}, 
is given by z* = ^ki, where 

k i = \{j€N i \x* j = l}\. 

It is obvious that ki E No- If fci is even i.e. an even number of 
variable nodes are set to 1 in the neighborhood of the check 
node i, then z* E No holds. Otherwise, Zi is an odd multiple 
of i. We then consider the Gomory cut for this row i. 

For the optimal solution y* we can partition P into a 
basis submatrix Pg and a non-basis submatrix Pn, i.e. 
P = [Pb Pn]- Let B and denote the index sets of the 
columns of P belonging to Pg and Pn, respectively. An 
(m + n + A) x (to + n + A) basis matrix, Pg, corresponding 
to the optimal solution y* can be constructed as follows. First 
we take the columns e 1 , . . . ,e m which are the identity vectors 
corresponding to the variables {zi . . . z m } into Pb- Secondly 
for j — 1, . . . , n, we include the column P m+ i if x* — 1 or 
pm+n+j s * = 1 jjj p B There exists n such columns since 

n 

i=i 

must hold due to (f7]). Finally we take the columns 
e m+2n+1 , . . . , e m + 2ll + A corresponding to the slack variables 
which are written for the iteratively added constraints. The 
variables corresponding to the columns in the basis matrix are 
called basic variables. The remaining columns of P form the 
non-basis submatrix Pn- The columns of Pn are the columns 
P m+J , j = l,...,n, for which x* = and the columns 
e m+n+j j _ i ^ n> f or which s* = 0. The variables 
corresponding to the columns in Pn are called non-basic 
variables. 

The Gomory cut for row i of P is given by the inequality 

^2 ^P lh ~ \Pih\)Vh > {Qi ~ [q~i\) (13) 

heN 

where p ih = (P B 1 ) l ■ (Pn) H , and ^ = (P B 1 ) i -q. Note that in 
our case i < to since only z* has non-integral components. In 
the following we investigate the structure of {Pg )i, (Pat)' 1 , 
p lh and qi. 

For a fixed i, it can easily be verified that the entries 

{P B ~ 1 )u, I = 1. . . . , to + n + A of {Pb~ 1 )i are given as 

!1, if Z = i 
l = m + j,j = 
otherwise 

(This can be verified by observing the changes on row i when 
we append an (m + n + A) x (to + n + A) identity matrix to 
Ps and perform the Gauss-Jordan elimination on the appended 
matrix in order to get Pg .) 
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Having found (P B 1 )i, is then computed by 



9 E « 



j:x* — l 



1 



(14) 
(15) 

(16) 



Thus, we showed that is | times the number of basic x 
variables in row i. Since Zi is not integer, the number of basic 
x variables in row i is odd. It follows that in our case the right 
hand side of the Gomory cut, q\ — |_SiJ> * s always \< 

Next, we compute pih = (Pg 1 )* ■ (Pn) H - The columns 
of Pat are the columns of P corresponding to non-basic x 
components (i.e. x* = 0) and non-basic s components (i.e. 
s*j = 0) j = 1,. . .,n. If (Pat)' 1 = P m+J such that x* = 0, 
then for a fixed value of h, the entries of (P/v) ?i , (Pn)oIi, 



n + A are given as 



(P, 



iV 



i if P 

1 , if o — to + j 
otherwise. 



1 and o < to 



If {P N ) h = P m+J such that s* = 0, then (P N ) h is the unit 
vector e m+ i . 

For the case that (Pn) K = P m+J where x* — 0, the only 
position where both (P^ 1 ); and (Pn) 11 may have nonzero 
entries is position i. For all other positions 1 = 1,..., m+n+A 
and I ^ j either (Pg 1 )u = or (Pn)iii = 0. This implies 



Pih = (P B ~ 1 )i(PN) h = 



if Pih = 1 
if Pih = 0. 



For the case that (P, 



iV 



P m +J where 



0, position 



-j is the only position where both (Pg 1 )i and (Pn) 11 may 
have a nonzero entry. This means, pih — (Pg 1 )i(P/v)' 1 = 
i for all non-basic s variables corresponding to the basic x 
variables in row i. If we denote the non-basic x variables in 



row i with the index set iV, \ S := {j 



: x. 



0} and the 



non-basic s variables corresponding to the basic x variables 



in row i with the index set S 
the Gomory cut as 



{7 



0}, we can write 



E 



'ih 



1 

\jpih\)yh > 2 



E 

jeNi\s 

E 1 

E x ^ 



X i 



^2' 

j&s 



E 



1 

> - 
~ 2 



> 



efficiently derive Gomory cuts with the following algorith m. 



■5^(1 - Xj )>l. (17) 
Since inequality (fTTT i is the forbidden set inequality obtained 



from the configuration S :— {j £ Ni 
the proof. 



1 } this concludes 

□ 



Cut Generation Algorithm 1 

Input : (x* , z*) such that x* integral, z* non-integral. 
Output : Gomory cut(s). 

1 : Set i = 1. 

2 : If ki = 2z* is odd go to 3. Otherwise go to 5. 

3 : Set configuration S := {j G Ni | x* = l}. 

4 : Construct constraint (0J. 

5 : If i < m, set i = i + 1 go to 2. Otherwise terminate. 



This algorithm has a computational complexity of (9(m(5 max ) 
because at most m values have to be checked until a violated 
parity check constraint is identified and 0(S max ) is the 
complexity of constructing (@). An algorithm to check if any 
forbidden set inequality is violated is also given in |fT3l . In 
order to find a violated forbidden set inequality, the algorithm 
of Taghavi and Siegel first sorts x. Next, at most S max 
Forbidden Set Inequalities have to be generated and validated. 
Repeating this procedure for to check nodes leads to an 
algorithm of time complexity 0(mS max +nlogn). In contrast, 
we can efficiently determine the violated parity checks using 
the indicator variables z. Having identified a violated parity 
check constraint i (if there exists any) we construct ((4]) easily 
by setting the coefficient of Xj for {j £ Ni : Xj = 1} to 
+1, the coefficient of Xj for {j G Ni : x* — 0} to —1 and 
\S\=h. 

Next we consider the situation that < x* < 1 for some 
j G J. Although it is still possible to derive a Gomory cut, 
Cut Generation Algorithm 1 is not applicable since 
Theorem 13. II holds only for integral x* . For non-integral x* 
we propose the following separation method in order to find 
valid cutting inequalities, the CUT GENERATION ALGORITHM 
2. The idea behind Cut Generation Algorithm 2 is based 
on Proposition 13.21 and Proposition 13.31 

Proposition 3.2: The Forbidden Set Inequalities derived 
from row i, i G {1, . . . , to}, of a parity check matrix H and 
the inequalities < x < 1, completely describe the convex 
hull conv(Ci) of the local codeword poly tope Cj. 
Proof: This is shown in Theorem 4 in |10|. □ 

Proposition 3.3: Let x* be a non-integral optimal solution 
of RIPD and x* £ conv(Ci). Then there are at least two indices 
j,k G J such that < xj < 1 and < Xk < 1. In other 
words check node i cannot be adjacent to only one non-integral 
valued variable node. 

Proof: If x* G conv(Ci) then it can be written as a convex 
combination of two or more extreme points of conv(Ci). 
Next we make use of an observation given in the proof of 
Proposition 1 in J8). Assume that check node i is adjacent to 
only one non-integral variable node. This implies that there 
are two or more extreme points of conv(C;) which differ in 
only one bit. Extreme points of conv(Ci) differ however, in 
at least two bits since they all satisfy parity check i which 
contradicts the assumption. □ 



Given an optimal solution of RIPD, (x* , z*) with x* G A given binary linear code C can be represented with some 
{0, 1} for all j £ J and z* G M\Z for at least one i G / we can alternative, equivalent parity check matrix which we denote 
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with H. Any such alternative parity check matrix for C is 
obtained by performing elementary row operations on H . Note 
that Proposition 13.21 is valid for any H. Likewise Proposition 
[33] holds as well for the parity check nodes i £ {1, . . . , m} of 
the Tanner graph representing H. The rows of H may also be 
interpreted as redundant parity checks. Given a non-integral 
optimum x* of RIPD, in CUT GENERATION ALGORITHM 2 
we search for a parity check which is adjacent to only one non- 
integral valued variable node. If we find such a parity check 
we know due to Proposition 13.31 that x* can not be in the 
convex hull of this particular parity check. Furthermore due 
to Proposition l3. 21 there exists a forbidden set inequality which 
cuts off x* . Note that in an exhaustive search algorithm one 
would check 2 m redundant parity checks if the parity check 
is adjacent to only one non-integral valued variable node. 

Instead of a computationally expensive exhaustive search 
we propose the Construct H Algorithm which resem- 
bles Gaussian elimination. We transfer matrix H into an 
equivalent matrix H by elementary row operations (adding 
two rows is in GF(2)). Our aim is to represent code C 
with an alternative parity check matrix H, so that in row 
Hi., there exists exactly one j £ J where Hij = 1 
and x* is non-integral. For all other indices h £ J \ {j} 
with H l . h = 1, x* h is integral. The CONSTRUCT H Algo- 
RlTHMtries to convert columns j of H with x* ^ Z into unit 
vectors. Note that at most m columns of H are converted. 



Construct H Algorithm 

Input : (x*,z*) such that x* non-integral 
Output : H. 

1 : Set I = 1, j = 1. 

2 : If x* E (0, 1) then go to 3. Else go to 4. 

3 : If I < m then do elementary row operations until 
Hij = 1 and H h3 = for all i 6 / \ {/}. Set I = I + 1. 

4 : Set j 
terminate. 



j + 1. If j < n then go to 2. Otherwise 



H can be obtained in 0(m 2 n). The CONSTRUCT H ALGO- 
RITHM is useful in the following sense. Suppose i £ / is a 
check node adjacent to several variable nodes j € J such 
that x* is non-integral. If H has such a row i then we use 
Proposition 13.21 and Proposition 13.31 to construct Forbidden 
Set Inequalities which cut off the fractional optimal solution. 
Specifically we construct the inequalities ( fl9] l or ( f20b . We refer 
to these inequalities as new Forbidden Set Inequalities. Note 
that Ni in the original H matrix and 2Vj in H are different 
index sets. First we calculate 

h= \{h£Ni\x% = l}\. (18) 

If ki is odd we use the inequality 

^2 x h -Xj - ^2 ^h<h- 1, (19) 

h£Ni-.xt=l heNi-.x*=0 

6 h L h 

otherwise, ki is even, i.e. 

^ x h + Xj- ^ x h<h- (20) 



Theorem 3.4: Let (x*,z*) E R" x M m be the optimal 
solution of the current RIPD formulation such that x* is non- 
integral. If there exists a Hi., such that Hi.j = 1 and x* is 
non-integral for exactly one j G J then the new forbidden set 
inequality is a valid inequality which is violated by x*. 
Proof: We have to show that: 

1) For ki odd [even] the inequality ( fl9l ) [ (|20[) ] is violated 
by x*. 

2) For ki odd [even] the inequality ( fl9] > [ (|20|) ] is satisfied 
for all x e C. 

Let i 6 / be a row of the reconstructed matrix H. We 
obtain i by performing elementary row operations in GF(2) 
on the rows of the original H matrix. Therefore it holds that 
Hi .x = mod2 for all x £ C. We show the proof for fc, odd. 
When ki is even the proof is analogous. 

1) Let ki be an odd number. For x* , since < x* < 1 the 
left hand side of ( [19] ) is larger than the right hand side thus 
x* violates ( fl9b . 

2) Suppose ki is odd and a;* is the optimal solution of RIPD. 
Our aim is to show that ( fT9l ) is satisfied by all codewords 
x £ C. First we define 



Si(x) 



Next we rewrite ( [T9i l as 



< fcj — 1 where Oj 6 {—1, !}• 



(21) 



jew. 

We also define the index sets 



5+ = {j G Ni : a,j = 1} with = ki 
S~ = {j e Ni : cij = -1} with \S~\ = 



Ni 



kj. 



Case 1 For any x G C it holds that &i(x) < fe, — 1: 
flj^j < fcj — 1 is fulfilled. 

Case 2a For any x G C it holds that £j (a;) > ki + 1: At most 
fcj of indices j G iV, where Xj — 1 can be in S + . Thus there 
is at least one index j G Ni with a;j = 1 in S~. Consequently 

Z a J x J - fcl ~ L 



Case 2b For any x G C it holds that <5j(a;) = feji If there is 
at least one index j G S~ with xj = 1 then 



E 



1. 



1 are in 5 f+ . Then for 



Otherwise all j G iV, with xj 
row i, ifjj.a; = 1 mod2 since ki is odd and therefore the 
contradiction x ^ C. □ 

Note that it is possible that each row of H has at least 
two j £ J such that Hi j = 1 and x* is non-integral. In this 
case no new forbidden set inequality can be found using CUT 
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The complexity of Cut Generation Algorithm 2 is in 
0(mn) since in the worst case each entry of H has to be 
visited once . 

We are now able to formulate our separation algorithm. In 
the first iteration, x* can be found by hard decision decoding. 
In all of the following iterations RIPD does not necessarily 
have an optimal solution with integral x*. If the vector (x* , z*) 
is integral then the optimal solution to IPD is found. If x* 
is integral but z* is non-integral we apply CUT GENERA- 
TION Algorithm 1 to construct Forbidden Set Inequalities. 
Although adding any forbidden set inequality suffices to cut 
off the non-integral solution (x* ,z*) we add all Forbidden 
Set Inequalities induced by all non-integral Zj based on the 
thought that they may be useful in future iterations. If x* is 
non-integral we first employ the CONSTRUCT H ALGORITHM 
. Then we check in Cut Generation Algorithm 2 if there 
exists a row Hi such that there exists exactly one j G J where 
Hij = 1 and x* is non-integral. If such a row does not exist, 
then the Cut Generation Algorithm 2 outputs an error. 
Otherwise we know from Theorem 13. 41 that there exists a new 
forbidden set inequality which cuts off x*. In H there may 
exist several rows from which we can derive new Forbidden 
Set Inequalities. In this case we add all new Forbidden Set 
Inequalities to the formulation RIPD with the same reasoning 
as before. The New SEPARATION ALGORITHM stops if either 
(a;*, z*) is integral which leads to an ML Codeword or CUT 
Generation Algorithm 2 returns an error which means 
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Two strategies which may be used in the implementation of 
the New Separation Algorithm are: 

1) Add all valid cuts which can be obtained in one iteration. 

2) Add only one of the valid cuts which can be obtained 
in one iteration. 

There is a trade-off between Strategies 1 and 2, since 
strategy 1 means less iterations with large LP problems and 
Strategy 2 means more iterations with smaller LP problems. 
We empirically tested Strategies 1 and 2 on the three codes 
described in the follwing section. For all the three codes 
Strategy 1 outperformed Strategy 2 in terms of running time 
and decoding success. 

IV. Numerical Results 

We compare the communication performance of our separa- 
tion algorithm with the standard LP decoding 1101 . BP decod- 
ing, and the reference curve resulting from ML decoding. The 
latter results from modeling and solving IPD using CPLEX 
9.120 |6| as the IP solver. These four algorithms, LP decoding 
(by Feldman et al. or Taghavi et al.), BP, New SEPARATION 
Algorithm , and ML Decoding(IP, CPLEX) are tested on 
two LDPC (one regular and one irregular) and one BCH code 
considering transmission over Additive White Gaussian Noise 
(AWGN) channels. Additionally we present for our separation 
algorithm the min, max and average values for the number 
of iterations, the number of generated Gomory cuts and the 
number of generated RPC cuts in tables I, II, III. We selected 
the (64,32) irregular LDPC code, Tanner's (155,64) group 
structured LDPC code [20| and the (63, 39) BCH code for 
our tests. The first LDPC code is constructed with Progressive 
Edge Growth algorithm. Tanner's (155, 64) LDPC code, which 
has minimum distance of 20 and girth of 8, is constructed as 
described in ll20l . The Frame Error Rate (FER) against signal 
to noise ratio (SNR) measured in Es/Nq is shown in Figures 
CD to [3 We used 200 iterations for BP decoding of (64, 32) 
irregular LDPC and Tanner's (155, 64) LDPC code. 

Figure [T] shows the results for the irregular (64,32) LDPC 
code with degree distribution Q /p,3,5,6] = [h = \,h = 
\,h = gj/e = §], 9[6] = [!]• Our separation algorithm 
performs by roughly O.bdB better than LP decoding for this 
LDPC code. It is important to note that the communication 
performance of the New SEPARATION ALGORITHM is supe- 
rior to the BP algorithm here. 

The results for the Tannner's (155,64) LDPC code are 
plotted in Figure [2] Performance of the BP and standard 
LP decoding is very similar in this case whereas the NEW 
Separation Algorithm gains around OAdB compared to 
both. It is worthwhile mentioning that BP decoding and 
our separation algorithm have a performance degradation of 
> 0.8dB compared to ML decoding for this group structured 
LDPC code. 

LP decoding via Forbidden Set Inequalities introduced in 
ifTOl cannot be used for high density codes since the number 
of constraints is exponential in the check node degree. This 

'irregular LDPC codes are described by variable node degree distribution 
fi and check node degree distribution gi, where /j and gi represents the 
fraction of variable nodes and check nodes with degree i respectively. 



no 



further cuts can be found. 



New Separation Algorithm . 

Input : Cost vector c, matrix H. 
Output : Current optimal solution x* . 

1 : Solve RIPD. 

2 : If the optimal solution (x* , z*) is integral then go 
to 6. Otherwise go to 3. 

3 : If x* is integral, then call Cut Generation Al- 
gorithm 1. Add the constraints to formulation RIPD, 
go to 1. If x* is non-integral go to 4. 

4 : Call Construct H Algorithm . Go to 5. 

5 : Call Cut Generation Algorithm 2. If the output 
is error then go to 6. Otherwise add the new constraint 
to formulation RIPD, go to 1. 

6 : Output x* and terminate. 



Cut Generation Algorithm 2 

Input : Optimum of RIPD s.t. x* non-integral, H. 
Output : New forbidden set inequality or error. 

1 : Set i = 1. 

2 : If there is exactly one j € J such that Hi j = 1 and 
x* G (0, 1), then calculate fc, and go to 3. Else go to 4. 

3 : If ki is odd [even] construct ( [T9b [ ((20]) ]. Terminate. 

4 : Set i = i + 1. If i < to then go to 2. Else output 

error. 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



X 



causes a prohibitive usage of memory in the phase of building 
the LP model. The adaptive approach of |fl3l overcomes this 
shortcoming and yet performs as good as LP decoding (see 
Section III). Therefore we used this method in the comparison 
of algorithms when decoding a dense (63,39) BCH code. The 
results for this code are shown in Figure [3] It should also 
be noted that BP decoding does not work for this type of 
codes due to the dense structure of their parity check matrix. 
Our approach is one of the first attempts (see |9|) to decode 
dense codes using mathematical programming approaches. 
Although the gap between ML decoding and our separation 
algorithm increases to roughly ldB, the results obtained by 
our algorithm are substantially better (more than 2dB) than 
the results obtained by adaptive LP decoding. 

To summarize, our separation algorithm improves LP decod- 
ing significantly for all three test setups. This improvement is 
due to new Forbidden Set Inequalities found by Cut GENER- 
ATION Algorithm 2. The constraints added by this algorithm 
are based on the rows of the alternative representations of the 
H matrix. These rows can also be interpreted as redundant 
parity checks. Consequently, the family A of inequalities we 
use includes a subset of the Forbidden Set Inequalities which 
can be derived from redundant parity checks and A is larger 
than the original family of Forbidden Set Inequalities. 

Regarding the complexity of the New Separation Al- 
gorithm , we present the minimum, average, and maximum 
number of iterations, cuts introduced by the Cut Genera- 
tion Algorithm 1 (shown in Gomory cuts column) and the 
number of cuts introduced by the Cut GENERATION ALGO- 
RITHM 2 (shown in RPC cuts column) in the tables U [II] and 
Hirjfor the codes (64, 32), (155, 64), and (63, 39) respectively. 
Note that the number of iterations can be considered as the 
number of times we call the LP solver. 




2 2.5 3 3.5 4 4.5 5 

SNR [dB] 



Fig. 1. Decoding performance of an irregular LDPC code (64,32). 

V. Conclusion 

In this paper we proposed a new IP formulation and its LP 
relaxation. Instead of solving the optimization problem, we 
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10 -' ^ — i _^ — i _^ 

2 2.5 3 3.5 4 

SNR [dB] 



Fig. 2. Decoding performance of Tanner's (155, 64) LDPC code. 




2.5 3 3.5 4 4.5 5 

SNR [dB] 



Fig. 3. Decoding performance of a BCH code (63,39). 



solve the separation problem. The indicator variables z yield 
an immediate recognition of parity violations and efficient 
generation of cuts. We used on one hand the Forbidden Set 
Inequalities of 1 1 1 which are a subset of all possible Gomory 
cuts. On the other hand we showed how to generate efficiently 
new cuts based on redundant parity checks. Note that the 
rows in our H matrix can be considered as redundant parity 
checks. It is known that RPC cuts improve the LP decoding 
via tightening the fundamental poly tope iflOl . |[T3l . However 
RPC generating approaches known to us cannot verify if the 
particular RPC really introduces a cut or not. Another open 
question addresses the configuration S to be used for the RPC. 
In our approach, once we ensure that there is only one j € Ni 
with non-integral x* in row H^., we can immediately find 
the configuration S and thus the new forbidden set inequality 
( fl9b or ( 1201 ). Additionally, Theorem 13.41 states that the new 
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Number of LPs solved 


Number of Gomory cuts 


Number of RFC cuts 


SNR 


Min 


Average 


Max 


Min 


Average 


Max 


Min 


Average 


Max 


1.8 


2 


5.942 


20 


2 


21.296 


42 





33.619 


207 


2.2 


1 


4.896 


21 





19.187 


41 





21.465 


227 


2.6 


1 


4.196 


19 





17.569 


42 





13.138 


177 


3.0 




3.48 


16 





15.07 


40 





6.895 


180 


3.4 




3.005 


19 





13.228 


39 





2.917 


145 


J.O 




2.725 


12 





11.254 


36 





1.513 


119 


4.2 




2.446 


11 





9.738 


31 





0.428 


111 


4.6 




2.297 


10 





8.195 


32 





0.27 


52 


5.0 




2.134 


6 





7.055 


31 





0.079 


25 


5.4 




1.977 


6 





5.585 


23 





0.014 


6 


5.8 




1.872 


6 





4.448 


18 





0.012 


12 



TABLE I 

Iterations and cuts derived for (64,32) LDPC code. 





Number of LPs solved 


Number of Gomory cuts 


Number of RPC cuts 


SNR 


Min 


Average 


Max 


Min 


Average 


Max 


Min 


Average 


Max 


2.0 


2 


6.093 


20 


20 


60.235 


94 





74.161 


594 


2.2 


2 


5.343 


22 


19 


57.148 


100 





48.667 


595 


2.4 


2 


4.828 


21 


19 


54.013 


94 





31.713 


640 


2.6 


2 


4.363 


23 


14 


50.817 


92 





20.254 


549 


2.8 


2 


3.954 


18 


15 


47.265 


96 





12.65 


468 


3.0 


2 


3.798 


26 


16 


45.324 


98 





10.776 


632 


3.2 


2 


3.47 


17 


16 


42.2 


79 





4.211 


431 


3.4 


2 


3.158 


19 


11 


38.381 


81 





1.293 


508 


3.6 


2 


3.13 


13 


6 


36.478 


76 





1.122 


228 


3.8 


2 


2.911 


10 


3 


34.085 


76 





0.324 


252 


4.0 


2 


2.81 


12 


7 


31.529 


66 





0.298 


238 


4.2 


2 


2.725 


9 


7 


29.576 


68 





0.146 


78 



TABLE II 

ITERATIONS AND CUTS DERIVED FOR (155, 64) TANNER CODE. 





Number of LPs solved 


Number of Gomory cuts 


Number of RPC cuts 


SNR 


Min 


Average 


Max 


Min 


Average 


Max 


Min 


Average 


Max 


2.4 




10.186 


24 





24.993 


56 





64.173 


200 


2.8 




8.802 


21 





23.464 


57 





50.382 


175 


3.2 




7.649 


22 





22.083 


53 





39.76 


180 


3.6 




5.911 


22 





19.401 


63 





25.184 


175 


4.0 




4.967 


21 





17.743 


54 





17.729 


179 


4.4 




4.111 


20 





15.379 


60 





11.612 


176 


4.8 




3.249 


18 





12.941 


59 





6.508 


177 


5.2 




2.703 


18 





10.944 


43 





4.002 


143 



TABLE III 

ITERATIONS AND CUTS DERIVED FOR (63,39) BCH CODE. 



forbidden set inequality is a valid inequality which cuts off 
the fractional optimal solution (x*,z*). 

These theoretical improvements are supported with empir- 
ical evidence. Compared to state of the art (adaptive) LP 
decoding our algorithm is superior in terms of frame error rate 
for all the codes we have tested. Moreover, it is competitive to 
the results obtained by BP decoding. In contrast to the latter, 
our approach is applicable to codes with dense parity-check 
matrix and offers a possibility to decode such codes. 

One future research direction is to find new cut families 
when Cut Generation Algorithm 2 stops. The polyhe- 
dral structure of the ML decoding will be further investigated. 
This will yield a branch-and-cut algorithm which we expect 
to further extend the applicability of our approach. 
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